CfBT^ 

Education Trust 


Paper presented at the 
Third International NLP Research Conference 
Hertfordshire University - 6 th and 7 th July 2012 
httD://nlpresearchconference. squaresoace. com/ 

For further information about this research contact r. churches@surrey. ac. uk ; 

rchurches(a)cfbt. com 


Training in influencing skills from neuro-linguistic programming (modelled 
from hypnosis and family therapy), in combination with innovative maths 
pedagogy, raises maths attainment in adult numeracy learners 

Allan, F., Bourne, J., Bouch, D., Churches, R., Dennison, J., Evans, J., Fowler, J., Jeffers, 
A., Prior, E. and Rhodes, L." 1 


Abstract 

Case study research suggests that NLP influencing strategies benefit teacher effectiveness. 
Maths pedagogy involving higher-order questioning, challenge, problem solving and 
collaborative working may be a way of improving attainment in adult numeracy learning, 
however, such strategies may be less effective if the relationship between teacher and 
learner does not reflect sensitivity to attitudes, beliefs and emotions (areas in which 
advocates of NLP claim effectiveness). The present study investigated these claims and the 
combined effect of such approaches using a pre- and post-treatment test design with 173 
adult numeracy learners. Teachers were randomly allocated to three conditions, these 
were: (1) teachers given no training (control condition); (2) teachers trained in innovative 
maths pedagogy (including more frequent higher-order questioning, challenge, problem 
solving and collaborative learning); and (3) teachers trained in both NLP and the innovative 
maths pedagogy. NLP training included suggestion using language patterns modelled from 
hypnosis, body language modelled from family therapy and spatial anchoring for emotional 
state management. A significant within-subject mean difference in maths test scores for the 
innovative maths pedagogy group (MD= 10.97, t(66) = 7.292, p< .0005, r| 2 =.446) was nearly 
twice that of the control ( MD= 5.67, t(42) = 3.099, p= .003, r\ 2 = .186). Although an 
attainment gap between pre- and post- treatment scores for the innovative treatment group 
alone (without NLP) appeared to close over time post hoc between-group contrasts 
indicated differences between pre- and post-treatment means were not statistically 
significant {p = .404 and p = .689, respectively). However, with the addition of NLP training, 
post hoc contrasts showed mean maths attainment had significantly improved compared to 
the control {p = .040) with mean difference, pre- and post-treatment attainment, increased to 
over three times that of the control ( MD= 18.35, t(62) = 9.552, p< .0005, r| 2 = .595). CfBT 
Education Trust carried out this research with funding from the Learning and Skills 
Improvement Service. 
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Introduction 


Neuro-linguistic programming (NLP) publications frequently claim to have modelled the 
subjective experience of highly able people (particularly in relation to communication skills) 
in a way that enables the transfer of effectiveness both within and across disciplines (see 
Tosey and Mathison (2009) for an academic appraisal of the field and its development). The 
earliest NLP publications originated when Richard Bandler and John Grinder (the co- 
founders of NLP) were a student and associate professor of linguistics at the University of 
Santa Cruz in California in the mid-1 970s. Their early books document the use of the NLP 
process of modelling with hypnotherapists and family therapists such as Milton Erickson and 
Virginia Satir (Bandler and Grinder, 1975a; b; 1979; Bandler, Grinder and Satir, 1976; 
Grinder and Bandler, 1976). These contain the first publication of their ideas about 
language (verbal and non-verbal), NLP approaches to the investigation of subjective 
experience and the internal mental processes which people are capable of perceiving. The 
first book to discuss teachers and classroom practice with NLP appeared in 1982 (Harper, 
1982), although an earlier publication looked at the development of self-esteem with children 
and teenagers (Anderson, 1981). Robert Dilts then produced a book which contained a 
specific chapter on NLP in Education (Dilts, 1983) originally written in 1981. Also in that 
year, Sidney Jacobson published the first of three extensive volumes on NLP and education 
(Jacobson, 1983; 1986a; b). Over the next 20 years, there have been over 20 more 
publications (see Carey et al. 2009 for a detailed review). Drawing on the evidence in 
relation to communication skills and teaching (for example, Muijs and Reynolds, 2005) NLP 
skills have, in recent years, been increasingly associated with the teacher effectiveness 
(Churches and West-Burnham ,2008; 2009; Carey et al., 2009; 201 1 ; Vieira and Gaspar, 
2012 ). 

From an academic perspective, there has been an increasing interest in research into NLP 
in recent years with calls for academic commentary to become more evidence based (Tosey 
and Mathison, 2009) rather than speculative and theoretical. In education, there has been a 
significant growth in the publication of research evidence on NLP, and in 2010, CfBT 
Education Trust published the first systematic literature review (Carey et al., 2010). This 
reviewed the content of 1 1 1 references including studies that contain research evidence. 

The authors identified 52 papers that claimed to contain confirmatory research evidence 
from 27 qualitative, 7 mixed-method and 18 quantitative education related studies. The 
review identified six quantitative studies that claim disconfirmatory evidence. None of the 
disconfirmatory studies was specifically in the area of adult numeracy learning, or classroom 
based mathematics teaching in general. There were no large-scale classroom based 
randomised controlled trials. Prior to the carrying out of the present research, a further 
search of the same databases used by Carey and colleagues in 2009 was undertaken. 
Between 2009 and 201 1 an additional 10 education related papers referencing NLP were 
published (el Gany et al., 201 0; Carey et al., 201 1 ; Jones, J, 201 0; Kudliskis, 2009; Mohsin, 
2009; Ran, 2009; Salmas- Villarreal, 2010; Saunders, 2009; Sibley, 2009; Slater et al., 2009; 
Tosey and Mathison, 2010). None of these contained evidence about the effectiveness of 
NLP in adult numeracy, or mathematics teaching specifically, however, two studies 
(Pishghadam et al., 2011a; b) report statistically significant positive correlations between 
teacher success in English language teaching and NLP. 

With the exception of the NLP Spelling Strategy (Malloy, 1 989; Malloy, 1 995), most of the 
evidence in support of using NLP in education suggests benefits in relation to areas like 
effective communication, engagement, questioning and classroom climate rather than 
specific classroom pedagogy. In a perspectives paper on the potential of NLP in education 
Churches and West-Burnham (2008), for example, associate the potential benefits of NLP in 
teaching with ideas about emotional climate and the importance of this for effective learning. 



Of particular relevance to this study are the 24 teacher-led action research case studies 
published by Carey and colleagues (2009; 2011). The peer reviewed journal paper (Carey 
et al., 2011) notes the potential benefits of teachers learning language patterns modelled 
from hypnosis, body language and emotional state management techniques but 
acknowledges the limitations of evidence provided by small scale teacher-led action 
research. Specifically, Carey and colleague (2011), recommend a large-scale randomised 
controlled trial to explore the potential of such techniques. The present study aims to build 
on that recommendation. 

Three recent papers illustrate the need for more research into adult numeracy and effective 
pedagogy (McLeod and Straw, 2010; NIACE, 2010; NRDC, 2010). In terms of progress, 
adult numeracy still lags behind progress in literacy in England (NIACE, 2010). Some of the 
contributing factors to this are clear from a recent extensive literature review by the National 
Research and Development Centre for Adult Literacy and Numeracy (NRDC, 2010). This 
endorsed earlier findings that suggest adult numeracy teaching is both under-researched 
and generally lacking a strong theoretical basis (Coben, 2003). The CfBT Education Trust 
literature review into Adult Basic Skills also illustrates that there is only a limited research 
base currently in this area compared to other areas of adult learning, particular in relation to 
pedagogy (McLeod and Straw, 2010). The NRDC review looked at academic literature, 
practitioner-focused publications, government reports and large-scale representative 
surveys. The review noted the importance of areas such as students’ self-perceptions of 
their numeracy difficulties (DfES, 2003) and that the gap between assessed and perceived 
skills is even greater in numeracy than it is for literacy (Bynner and Parson, 2006). In 
relation to the climate that teachers create in their classrooms, teachers who are effective 
are able to motivate learners to persist (Lopez et al., 2007; Swain et al., 2005) and deal with 
the high levels of anxiety and fear that is felt by many adult numeracy learners (Sewell, 

1981 ; Meader, 2000). These anxieties often originate from early childhood and can be 
traumatic and long-lasting (Coben and Thumpston, 1 996). Anxiety may also weaken 
memory, logical thinking and the ability to work methodically (Ashcraft, 2002). The nature of 
the relationship between the learner and teacher itself may therefore have a significant 
impact with effective teachers being sensitive to the attitudes, beliefs and emotions of their 
learners (Cohen, 2005) and by extension it could be argued that adult numeracy teachers 
need the skills to deal with these areas effectively. In terms of pedagogy, research 
evidence suggests that engagement and making maths meaningful is important (Salford, 
2000; Baker, 2005) as is the teaching of abstract concepts not just basic numeracy (Swain 
et al., 2005). Salford, for example, drawing on Piaget argues for a constructivist approach in 
which learners should be allowed to work out the general rules of mathematics from 
exploratory situations (Salford, 2000). ‘Bad’ practice is seen as involving the application of 
procedures without understanding (Swain, 2005). High levels of effective questioning, 
collaboration and engagement in which learners are challenged to think for themselves may 
therefore be more effective (see Swain and Swan, 2007). In addition, the need to focus on 
the development of effective models for mathematics teaching rather than merely the 
identification and recruitment of the most mathematically talented is becoming increasingly 
clear across all education phases (Burghes and Robinson, 2009). 

Principles for more innovative and effective adult numeracy teaching have been defined by 
the National Centre for Excellence in the Teaching of Mathematics (NCETM, 2008). These 
build on earlier research by Swain and Swan (2007). These principles include the need to 
build on existing knowledge more effectively, expose learner misconceptions, increase the 
amount of higher-order questioning, more appropriate use of whole class, individual and 
small group work, encouraging reasoning rather than simplistic answering, the use of richer 
collaborative tasks and the need to make mathematics relate to the real world. The report 
also emphasised the appropriate use of technology, confronting difficulties rather than 
avoiding them, greater use of mathematical language and the need to ensure that learners 



understand how they have learned things as well as what they have learned. This said there 
has not been any controlled research to test these claims. 

It would appear, from a review of the literature, that achieving effective adult numeracy 
pedagogy might require the inclusion of training that supports teachers to develop more 
effective communication skills (in order to deal with issues of learners’ anxiety, fear and 
motivation) as well as the types of effective pedagogy described above. In such a context, 
communication skills usually found in therapeutic contexts might be useful. Formal clinical 
hypnosis is an increasingly established field (see Oakley and Halligan, 2009, for a review of 
the cognitive neuroscience evidence), however, it has also been demonstrated that 
hypnotic-suggestive communication can have an effect outside of formal hypnosis in 
contexts such as advertising (Kaplan, 2007). The best predictor of a person’s hypnotic 
suggestibility is individual responsiveness to the same suggestions outside of hypnosis 
(Braffman and Kirsch, 1999). Furthermore, an emerging consensus in the field of hypnosis 
(Kirsch et al., 2011) acknowledges that: all suggestions experienced following induction can 
be experienced without it; that hypnotic inductions only slightly increase suggestibility; and 
that waking and hypnotic suggestibility are highly correlated. In relation to NLP one recent 
study also suggested evidence for a relationship between some NLP techniques and 
hypnotizability. Research by Kirenskaya and colleagues (2011) showed a decrease in 
negative emotional intensity for both high and low hypnotizables but with autonomic activity 
(heart rate, skin conduction span) decline observed in high hypnotizables only. Image 
vividness and emotional intensity were also significantly higher in the high hypnotizability 
subjects. Whether NLP represents a sub-school of hypnosis, a set of techniques, or a field 
of study in itself remains a live debate (see Tosey and Mathison, 2009). This study has 
sought to integrate two domains: NLP influencing strategies; and innovative maths 
pedagogy including approaches such as higher levels of collaborative learning, challenge, 
engagement and higher-order questioning - both of which claim benefits, assess their 
combined effectiveness and the extent to which the NLP training might enhance effective 
pedagogy. Therefore, this research sought to contribute to both the debate about the 
usefulness of NLP in education and the effectiveness of NLP in general. 

The present study consisted of a research design with three between-subject conditions and 
a within-subject pre- and post-treatment maths attainment test. The three conditions were: 
(1) teachers given no training (control condition); (2) teachers trained in innovative maths 
pedagogy (involving higher amounts of higher-order questioning, challenge, problem solving 
and collaborative learning); (3) teachers trained in NLP influencing skills in addition to the 
innovative math pedagogy training in Condition (2). Each adult learner participant group 
took the same maths attainment test pre- and post-treatment. A review of the literature and 
subject matter content of NLP training suggested that NLP communication skills were 
unlikely to improve maths attainment in themselves if the quality of pedagogy being used 
was in question (because NLP communication skills are essentially content-free). Rather 
there was more likely to be a measurable effect if a baseline of good pedagogy was 
established and known to be in place. The study’s design therefore allowed for the testing of 
two hypotheses: 

Hypothesis (a) - adult learners whose teachers are trained in innovative maths pedagogy 
attain higher maths results than adult learners whose teachers have had no training 

Hypothesis (b) - training in NLP influencing skills enhances the maths attainment of adult 
learners whose teachers have trained in innovative maths pedagogy 


Method 



Participants 

Prior to participant recruitment a priori power analysis was carried out using G*Power 3.1 .2 
(Faul et al., 2007; 2009) in order to estimate a minimum sample size for the study. No prior 
studies were available on which to base an effect size estimate. An effect size of 0.1 , a = 
0.05, p-1 = 0.95 (repeated measures, within-between interaction) was used in the 
calculation. Results indicated a recommended sample size of >207. Anticipating substantial 
levels of participant attrition, because of the transient nature of adult numeracy classes, the 
recruitment approach aimed for a target number of 300-350. Recruitment was via e-mail 
and presentations at networking events and targeted the teachers of adult numeracy 
learners across the southeast of England. Initially, 37 Further Education sector teachers 
expressed interest in participating in the study, 24 teachers eventually took part with 278 
adult learners completing the initial baseline testing. Offender learning was not eligible 
because of data sharing issues. As anticipated, there were substantial levels of participant 
attrition during the study. In addition (to reduce the risk of ceiling effects), the decision was 
taken to remove learners who scored more than 95% in the initial pre-treatment maths test. 
Six learners whose scores were above this cut-off point were withdrawn from the participant 
group following the baseline test. In total, 173 adult learners between the ages of 16 and 70 
(M= 30.94, SD= 13.12) completed the study, 71 males and 102 females. The following 
number of participants completed the post-treatment maths test: no training, n = 43; training 
in innovative maths pedagogy, n = 67; training in NLP and innovative maths pedagogy, n = 
63 (see Table 1). 


Table 1 


Adult learner participants who completed the research study 


Condition 

Number of 
participants 

Age 

Age range 

SD 

Males 

Females 

No training 

43 

37.79 

16-68 

13.44 

11 

32 

Innovative 
maths pedagogy 

67 

27.09 

16-71 

12.37 

18 

49 

NLP and 
innovative 
maths pedagogy 

63 

23.44 

16-51 

10.61 

41 

22 

Total 

173 

30.94 

16-70 

13.12 

71 

102 


As an incentive, teachers whose classes completed the research could attend a post- 
research conference and additional training. There were no incentives for the adult learners. 
All participants received treatment that was in accordance with standard ethical research 
guidelines. Dr Paul Tosey, University of Surrey, was consulted about the ethical application 
of NLP within the research design. The Skills for Life Development Centre, University of 
Sussex Innovation Centre monitored and controlled all data security, entry and analysis. 


Materials 

All adult learner participants received the same pencil and paper single level maths test that 
they completed pre- and post-treatment. Participants also completed a demographic 
questionnaire (e.g. age, gender). The maths test was from the Department for Education 


and Skills ReadWritePlus Skills for Life Diagnostic tools in Numeracy Testing. The test 
covered curriculum areas as specified in the Skills for Life Adult Core Curriculum for 
Numeracy: Entry 1 , 2, 3 and Levels 1 to 3 (DfES, 2001). In addition, participants completed 
an attitude to maths learning questionnaire, repeated pre- and post- treatment. A further 
research paper (currently in draft) will discuss the results from this. 


Procedure 

Prior to the administration of the pre-treatment test and demographic questionnaire the 
teachers were randomly allocated to one of the three conditions described above, whilst 
controlling for a number of background factors to ensure a similar distribution in each 
condition. The controlled variables were teacher qualification level, number of years in 
teaching, spread of experience in teaching Skills for Life Numeracy, Functional Maths and 
Key Skills - Application of Number. No teachers from the same organisation/location 
completed the same condition to avoid the risk of ‘content sharing’ in relation to the training 
received and ‘contamination’ between participant groups. 

One criticism of NLP is that it is a form of ‘cargo cult’ psychology (Roderique-Davies, 2009). 
The implication being that any effects are perceptual (or placebo) and exist only in the minds 
of converts - although no research has tested this hypothesis yet. In response to this 
criticism, the present study implemented a number of additional controls. All the adult 
learners were kept ‘blind’ to the purpose of the study and to the content that their teachers 
had, or had not, been trained in - the teachers simply adapted their practice without making 
any explicit references to anything that they had learned. In relation to the teachers, the no 
training group remained ‘blind’ to the content that the other teacher groups had been trained 
in and the innovative maths pedagogy alone group remained unaware of the content of the 
training given to the NLP and innovative maths group. Furthermore, the NLP trained 
teachers did not know that they were to receive training in NLP until they arrived on the first 
day of the NLP training programme. All other participants remained unaware that NLP was 
part of the research design. No teachers whose learners completed the study had received 
any previous training in NLP. 

The adult learners gave consent before completing the pre-treatment maths test, attitudes 
questionnaire and demographic questionnaire. Teachers conducted the pre- and post- 
treatment maths attainment tests in their own classrooms in Further Education, sixth form 
colleges, work-based learning providers and adult and community learning providers. There 
was no time limit for the test. Learners could take as much time as they needed to attempt 
(in one session within one lesson) as many questions as possible before handing in the test 
paper. Pre-treatment tests took place in the middle to end of the autumn term and post- 
treatment tests at the end of the spring term/beginning of the summer term, although exact 
control of this variable was difficult because of the nature of adult numeracy learning and 
differences in weekly contact time and term dates. Where there was variation this was 
similar within each condition. Teachers themselves received instructions to avoid reading 
the maths test and simply to invigilate the test on the two occasions, collect it in and post it 
immediately to the project administrator. 


The three conditions were as follows: 


(1) No training 



Adult learners in this group completed the maths attainment test at the beginning of the 
allocated time-period and again at the end, their teachers received no training (from the 
project) between the two testing points. 


(2) Training in innovative math pedagogy (involving higher amounts of higher-order 

questioning, challenge, problem solving and collaborative learning) 

Adult learners in this group completed the maths attainment test at the beginning of the 
allocated time-period, their teachers then received 2 days of training in innovative maths 
pedagogy. The principles of effective adult numeracy teaching, as defined in the National 
Centre for Excellence in the Teaching of Mathematics 2008 report (NCETM, 2008), and 
which define a more innovative approach to pedagogy in adult learning, formed the basis for 
the curriculum, with a particular emphasis on higher-order questioning: 

• Building on existing knowledge 

• Exposing misconceptions 

• Using higher-order questioning 

• Appropriate whole class, individual and small group work 

• Encouraging reasoning rather than answer getting 

• Using rich collaborative tasks 

• Creating connection between topics, both within mathematics and to the real world 

• Using technology appropriately 

• Confronting difficulties rather than avoiding them 

• Developing mathematical language 

• Understand what has been learned and how 

The training the teachers received also built on ideas and approaches from the Maths4Life 
project (see Carpentieri et al. (2010), for a summary). Teachers in Condition (2) were 
encouraged to use the online adult numeracy resources available at the Learning and Skills 
Improvement Service Excellence Gateway throughout the research period. Participants also 
received non-NLP related mentoring support to help them to embed the training that they 
had received. This mentoring was carried out by the same mentors who mentored group 3 
(see below) all of whom were trained to NLP Diploma level but in the case of group 2 were 
briefed to avoid using any NLP related techniques. At the end of the allocated time-period, 
the adult learners completed the same single level maths test again. 


(3) NLP and innovative maths pedagogy group 

Adult learners in this group completed the maths attainment test at the beginning of the 
allocated time-period, their teachers then received the same innovative math pedagogy 
training as Condition (2) above. The teachers also received a further 4 days of training in 
NLP. The NLP training curriculum consisted of: 

• Learning to use influential language patterns modelled from hypnosis (the Milton 
model (Grinder and Bandler, 1975b; 1981)) in order to formulate positive suggestions 
in relation to attainment, motivation and behaviour (Churches and Terry, 2007). 
Specifically, the teachers were taught how to create positive presuppositions and 
suggestions and how to use: cause and effect and complex equivalence patterns, 
chained modal operators, double binds, embedded commands, linkage language, 
pacing and leading, universal quantifiers, yes set and yes tags. 

• Learning to understand the effects of Satir category body language (Blamer, 

Placater, Leveller, Computer, Confuser) and apply appropriate categories in a 



congruent way (Bandler and Grinder, 1976; Bandler, Grinder and Satir, 1976) whilst 
communicating in the classroom (Churches and Terry, 2007). This component of the 
training included foundation training in the development of sensory acuity and the 
use matching and mirroring to build rapport (Bandler and Grinder, 1979) 

• Learning to use anchoring to support emotional state management (Bandler and 
Grinder, 1979) whilst teaching using spatial anchoring (Churches and Terry, 2007) 

Details of the training protocol are available from nlpresearch@surrey.ac.uk . At the end of 
the allocated time-period, the adult learners completed the same single level maths test 
again. 

Teachers in Condition (3) received additional NLP reading material (Churches and Terry, 
2007; Terry and Churches, 2008) and the Teaching Influence cards used in previous NLP 
classroom case study research (Carey et al., 2010; 2011). They also received mentoring 
support from mentors trained to INLPTA NLP Diploma Level (INLPTA, 2005) to help them to 
embed the training they had received. 


Results 

Descriptive statistics show that the mean for the single level maths test improved from 60.37 
to 66.05 in the no training group; from 54.61 to 65.58 for innovative math pedagogy group; 
and from 56.49 to 74.84 for the NLP and innovative maths group (see Figure 1 below). 


Figure 1 


Maths attainment before and after treatment as a function of three conditions 



1 ( No training 

♦ Teachers trained in innovative 
maths pedagogy 

* Teacherstrained in NLP and 
innovative maths pedagogy 


To address hypotheses (a) and (b) paired-samples t-tests compared pre- and post-treatment 
adult learner maths attainment in the three conditions: no training, training of teachers in 
innovative maths pedagogy; training of teachers in NLP influencing strategies. Further 
analysis used a 3 x 2 analysis of variance with post hoc comparisons. Preliminary 
assumption testing used SPSS Explore. In relation to carrying out the paired-samples t- 
tests, pre- and post-treatment maths test within-subject data was satisfactory in relation to 
assumption of normality and the presence outliers. The final analysis applied post hoc tests 
suitable for use with unequal samples sizes. In relation to the assumptions required for the 


analysis of variance and between-subject post hoc comparisons, the pre-treatment data 
violated the assumption of homogeneity of variance, therefore the data was analysed using 
the Games-Howell multiple comparisons test. Post-treatment data was satisfactory in 
relation to all assumptions and was assessed using Dunnett’s t. 

Paired-samples t-tests showed that the improvement in maths attainment in Figure 1 was 
statistically significant for all three of the participant groups: no training, t(42) = 3.099, p = 
.003 (2-tailed); training in innovative maths pedagogy, t(66) = 7.292, p< .0005 (2-tailed); 
training in NLP and innovative maths pedagogy, t(62) = 9.552, p< .0005 (2-tailed). Partial 
eta squared values for all three conditions indicated they could be interpreted as having had 
a large effect (Cohen, 1998): control (no training), r\ 2 = .186; training in innovative maths 
pedagogy, ti 2 =. 446 ; training in innovative maths pedagogy and NLP, r\ 2 = .595. The largest 
statistically significant improvement in maths attainment was for the participant group whose 
teachers trained in both NLP influencing skills and innovative maths pedagogy. Specifically, 
the NLP and innovative maths pedagogy group had the highest mean increase of the three 
participant groups (MD= 18.35; SE = 1.92) with an increase in maths attainment that was 
nearly three times that of the control condition ( MD = 5.67; SE = 1 .83) and one and a half 
times that of the innovative maths pedagogy alone group {MD = 1 0.97; SE = 1 .50). As can 
be observed in the data above, and in Table 2 below, innovative maths pegagogy alone also 
improved attainment with a mean difference score that was nearly twice that of the control 
condition. 


Table 2 


Paired differences for the three conditions 



Mean 

Difference 

Standard 

Deviation 

Standard 

Error 

95% Confidence Interval of the 
Mean Difference 

Lower bound 

Upper bound 

Control group difference 
between pre- and post- 
maths test scores 

5.67 

12.01 

1.83 

1.98 

9.37 

Innovative maths 
pedagogy difference 
between pre- and post- 
maths test scores 

10.97 

12.31 

1.50 

7.97 

13.97 

NLP and innovative 
maths pedagogy 
difference between pre- 
and post- maths test 
scores 

18.35 

15.25 

1.92 

14.51 

22.19 


However, scrutiny of the confidence intervals for pre- and post-treatment means themselves 
(Figure 2 and Table 3 below) suggest the increase for the no treatment group would not 
generalise 95% of the time to the population because the post-test score for this group 
(66.05) lies within the 95% confidence interval for the pre-test mean (lower bound = 53.49, 
upper bound = 67.25). This is also the case for the innovative maths pedagogy alone 
results. By contrast, the confidence interval data from the NLP and innovative maths 
pedagogy groups indicates that the statistical model presented by the data is likely to 
represent true values for the population from which the samples were drawn. 


Figure 2 


95% confidence intervals around the pre- and post-treatment 
mean maths attainment scores for the three conditions 



No training 


Training in innovative Training in NLP and 
maths pedagogy innovative maths pedagogy 


Table 3 

Mean scores for the three condition pre- and post- tests with confidence intervals 



Maths 
attainment 
mean score 

Standard 

Deviation 

Standard 

Error 

Minimum 

Maximum 

95% Confidence 
Interval 

Lower 

bound 

Upper 

bound 

Pre- control 
condition 

60.37 

22.35 

3.41 

18 

95 

53.49 

67.25 

Post- control 
condition 

66.05 

22.02 

3.36 

16 

98 

59.27 

72 82 

Pre- innovative 
maths pedagogy 
training 

54.61 

23.54 

3.50 

10 

94 

49.03 

63.16 

Post- innovative 
maths pedagogy 
traininq 

65.58 

23.83 

3.50 

20 

99 

59 08 

73.12 

Pre- NLP and 
innovative maths 
pedaqoqy traininq 

56.49 

21.47 

3.02 

6 

95 

54.65 

66 84 

Post- NLP and 
innovative maths 
traininq 

74.84 

20.26 

3.49 

21 

99 

67.68 

81.72 


A 3 x 2 analysis of variance with repeated measures explored the trend seen in Figure 1 , 
above. The dependent variable was maths attainment. The first factor was the condition 
(no training; innovative maths pedagogy; NLP and innovative maths pedagogy). The 
second factor was the timing of the maths test (before or after treatment). Both the main 
effect of level and timing (F(1,170) = 126.37, p< . 0005) and the level x timing interaction 
were significant (F(2,170) = 12.02, p< .0005). In order to explore the meaning of the data 
post hoc analysis used a series of contrasts comparing the pre- and post-treatment 
interaction with the three conditions. The first set of contrasts explored the differences 



between the levels for the pre-treatment maths tests. Because sample sizes for the pre- 
treatment tests were unequal and Levine’s Test of Equality of Error Variance was significant 
the Games-Howell multiple comparisons procedure was used. All of the contrasts between 
the levels were not significant (control-innovative maths, p= .404; control-NLP and 
innovative maths, p = .647; innovative maths alone-NLP and innovative maths, p= .883). A 
further series of three contrasts explored the three post-treatment maths attainment scores 
after the treatment phase of the research used Dunnett’s t because Levine’s Test was not 
significant for this data. This test also allowed for the treatment of one condition as a control 
(the no training group) and for the comparison of other conditions against it. The contrast 
between the control condition and the NLP and innovative maths condition was significant 
{MD = 8.80, SE = 4.38, p = .040). As expected, from what can been seen in Figure 1 the 
contrast between the control condition and the innovative maths condition was not significant 
(p = .689). A final contrast with innovative maths pedagogy defined as the control, indicated 
a statistically significant difference between the NLP and innovative maths and innovative 
maths alone conditions {MD = 9.26, SE = 3.88, p = .01 7). 

A final test determined levels of significance in relation to individual participant improvement 
between pre- and post-treatment maths attainment (improvement was defined as ‘post- 
treatment test score’ minus the ‘pre-treatment score’ for each individual participant). As 
expected from the 3 x 2 analysis of variance the one-way analysis of variance produced the 
same significant main effect (F(2,1 70) = 1 2.02, p< .0005). Post hoc contrasts (Dunnett’s t) 
indicated a statistically significant difference between the NLP and innovative maths 
condition and the control ( MD= 12.67, SE = 2.65, p<. 0005) and between the innovative 
maths pedagogy and the control {MD = 5.30, SE = 2.62, p = .039). A further contrast with 
innovative maths pedagogy defined as the control showed a significant difference between 
this condition and the NLP and innovative maths group’s level of improvement ( MD= 7.38, 
SE = 2.35, p = .002). 


Discussion 

The current research intended to examine the extent to which innovative maths pedagogy 
(involving higher amounts of higher-order questioning, challenge, problem solving and 
collaborative learning) could improve adult numeracy attainment and whether training in NLP 
influencing skills could further enhance any improvements. A review of the literature showed 
that there had been no large-scale quantitative research studies in relation to either of these 
areas. For the purpose of this research, we wrote a formal NLP research training protocol. 
Future research could use this to standardise the pre-research training of teachers. 

The data clearly indicated that NLP training in additional to innovative maths pedagogy 
significantly enhanced attainment compared to both the control and innovative maths 
pedagogy condition. Fiowever, the extent to which innovative maths pedagogy alone 
contributed to improved maths scores was ambiguous. 

Hypothesis (a) - adult learners whose teachers are trained in innovative maths pedagogy 
attain higher maths results than adult learners whose teachers have had no training 

Interpretation of the data for the innovative maths alone condition is ambiguous and 
depends of the emphasis placed on each of the tests used. On the one hand, although 
training in innovative maths pedagogy alone appeared to have closed a small attainment 
gap between the pre-treatment maths attainment of the control condition (M = 60.37) and the 
innovative maths pedagogy group (M = 54.61), the between-subject post hoc tests for both 
the pre- and post-treatment maths attainment scores were not significant. Therefore, this 
difference could have occurred by chance. Confidence intervals around the means further 



challenge assertions about the closing of an attainment gap. An interpretation based solely 
on this data would result in an acceptance of the null hypothesis. However, there was also a 
statistically significant within-subject mean difference for the innovative maths pedagogy 
condition, nearly twice that of the control group. Again, interpretation of this result depends 
on the emphasis placed on different statistical data, in this case confidence intervals versus 
statistically significant within-subject effects. Finally, assessment of levels of improvement in 
innovative maths attainment, compared to the control, found a statistically significant result 
when the data was analysed from this perspective. Taking these things into consideration 
another interpretation could be a partial but not conclusive confirmation of hypothesis (a). 
Further, bearing in mind that participant attrition reduced the sample size for the control this 
will have affected the statistical significance of the difference between the pre-treatment 
innovative maths condition and the control and therefore the significance of any closing of 
attainment gap between the initially lower attaining innovative maths group. Taking this into 
account, and the two possible interpretations of the data, we have favoured the second 
interpretation in our conclusions. 

Hypothesis (b) - training in NLP influencing skills enhances the maths attainment of adult 
learners whose teachers have trained in innovative maths pedagogy 

The statistically significant results for the paired-sample t-tests, and the post hoc 
comparisons, confirm that teachers trained in NLP influencing strategies, in addition to 
innovative math pedagogy, improved maths attainment. This was the case in relation to 
mean difference between the pre- and post-treatment scores and in relation to the difference 
in final test score between the three conditions, a clear confirmation of hypothesis (b). 
Confidence intervals further reinforce an unambiguous interpretation. 


Possible methodological limitations and suggestions for future research 

Because NLP was not trained as a separate condition it is unclear as to the extent to which 
the improvements in maths attainment were completely due to the NLP training or the result 
of the combination of NLP and innovative math pedagogy. This said, it seems unlikely that 
training in content-free communication skills would improve an area of the curriculum like 
mathematics that has very clear subject and pedagogic requirements without a degree of 
good practice also being in place. Furthermore, the study did not seek to test whether NLP 
influencing skills would also improve maths attainment where teaching methods were more 
traditional. A second limitation is that the level of participant attrition reduced the number of 
participants in the control condition to below that of the other two participant groups and may 
have affected the results. Specifically, the lack of statistically significant evidence (in post 
hoc tests) that innovative maths pedagogy alone closed, what in terms of the descriptive 
statistics appeared to be a gap in attainment between the control and this group, could be 
the result of this lower sample size for the control condition. Thirdly, the teachers in the NLP 
and innovative maths group may have been more motivated to maintain the implementation 
of strategies during the research period between mentoring visits because they received 
more training days overall. Finally, the NLP training contained a number of components that 
some claim need integration for NLP to be successful as a communication strategy (see 
Churches and West-Burnham, 2009). This combination of strategies makes it impossible to 
assess the individual contribution of any of the distinct components. One further issue may 
have affected the difference between the control condition and the maths alone group as 
many of the maths pedagogy strategies that were included in the training have been 
gradually being implemented across the sector generally, making it difficult to assess the 
extent to which already existing skills may have confounded differences. 



A larger study might be able to produce clearer results in relation to innovative maths 
pedagogy’s contribution to improvements in maths attainment. There is also a need to 
explore the effectiveness of individual NLP components, compared to the combined 
effectiveness of techniques. Finally, future research might explore whether NLP can 
improve attainment irrespective of teaching method or could look at whether NLP had an 
equal effect on other maths teaching methods. 


Conclusions 

Overall, training teachers in content-free NLP influencing strategies modelled from hypnosis 
and family therapy appeared to improve maths attainment for adult learners where there was 
a baseline of innovative maths pedagogy involving higher amounts of active learning and 
group work in place. It was not possible to determine conclusively the effectiveness of the 
particular maths pedagogy used in the study, although there was partial evidence that it had 
a beneficial effect on its own. Based on our experience in this research, NLP training has 
the potential to improve attainment if used as an enhancement to the development of 
pedagogy and subject knowledge but not we would argue instead of such programmes. 
Ultimately, from a classroom perspective, the success of any content-free influencing 
strategy is most likely to depend on the quality of the content communicated (e.g. quality of 
subject knowledge (so important in mathematics teaching (see for example, Burghes, 2011)) 
and appropriateness of learning activities etc. However, this present study suggests that 
without effective communication good subject knowledge may not be enough. From a 
broader perspective, and in relation to the evidence from adult numeracy research, the 
evidence from this study supports earlier research that shows that engagement and a 
teacher’s ability to create positive classroom climate is at least as important as pedagogy 
(see Muijs and Reynolds (2005) for a discussion of the importance of classroom climate, 
generally). What this present study suggests, for the first time, is that some of the 
interpersonal and intrapersonal skills associated with creating the right conditions for 
effective adult numeracy may be able to be modelled, codified and transferred to and 
between teachers. In relation to criticisms of NLP generally (Heap, 1998; Heap 2008; 
Roderique-Davies, 2009) we believe that the effect sizes in this study can be interpreted as 
preliminary evidence that some NLP techniques, when applied to specific contexts, are more 
than ‘cargo cult’ psychology (Roderique-Davies, 2009). A conclusion reinforced by recent 
statistical research that has yielded confirmatory evidence of the positive effects of NLP in 
the fields of radiology (reducing the need for anaesthesia in claustrophobia patients 
undergoing MRI) (Bigley et al., 2010), psychotherapy (Stipancic, 2010) and in relation to 
high verses low hypnotizables (Kirenskaya, 2011). However, we accept that some of the 
criticisms aimed at the theoretical foundations of NLP, could be valid and agree with writers 
who suggest a reappraisal of some of the theoretical explanations in the NLP literature (see 
for discussions, Carey et al., 2009; Tosey and Mathison, 2010). 
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